Skip to content

fix(i18n): CI-enforced dictionary guard (FIX-I18N-DICT-GUARD) - #115

Merged
FullGas1 merged 4 commits into
developfrom
fix/i18n-dict-guard
Aug 10, 2026
Merged

fix(i18n): CI-enforced dictionary guard (FIX-I18N-DICT-GUARD)#115
FullGas1 merged 4 commits into
developfrom
fix/i18n-dict-guard

Conversation

@FullGas1

@FullGas1 FullGas1 commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Reported by FullGas: some F10 menu entries stay untranslated when switching CTLD's language to Korean.
  • Root cause: translate_i18n.py's stub detection only ever matched a non-EN value identical to the EN text; generate_i18n_dicts.ps1 -Apply writes a freshly-added non-EN entry as an empty string instead, so a new key was never picked up for translation, with or without ANTHROPIC_API_KEY set.
  • New CI job i18n-guard, modeled on changelog-guard and diff-scoped against the PR base: blocks unconditionally on a key missing from any of the four dictionaries, and blocks by default, bypassable via the skip-i18n label, on a newly-introduced empty non-EN entry.
  • Pre-existing debt already on develop - 91 CTLD_i18n_ko.lua plus 76 CTLD_i18n_es.lua empty entries - is explicitly out of scope; this guard only prevents new debt. A follow-up repayment lot must land next.
  • Full design rationale and rejected alternatives: ADR 0013, dev/adr/0013-ci-i18n-dict-guard.md.
  • Lot: .backlog/FIX-I18N-DICT-GUARD/ - PRD plus 3 tickets.

Test plan

  • pytest tools/build/ - 17 tests green locally, first coverage tools/build/ has ever had
  • check_i18n_diff.py manually verified: catches a simulated newly-added empty stub, reverted
  • generate_i18n_dicts.ps1 dry-run confirmed 0 MISSING on this branch, job would pass
  • translate_i18n.py sanity-run locally without ANTHROPIC_API_KEY, unchanged warning/exit-0 behavior
  • Both modified workflow YAML files validated with yaml.safe_load
  • CI green on this PR - i18n-guard, python-quality, existing jobs
  • Once merged, update .backlog/README.md index line to merged PR number

Generated with Claude Code

Summary by Sourcery

Add a CI-enforced guard to prevent new untranslated i18n menu entries and fix local stub detection so newly added dictionary keys are correctly identified for translation.

New Features:

  • Introduce the i18n-guard CI job that fails pull requests when new i18n keys are missing from any dictionary or when they add newly empty non-English entries, with a skip-i18n label as a controlled bypass for the latter.
  • Add the check_i18n_diff.py helper script to detect newly introduced empty non-English dictionary entries by diffing against the PR base.
  • Provide shared i18n_dict_utils.py helpers to consistently parse CTLD i18n Lua dictionaries across tools.

Bug Fixes:

  • Correct translate_i18n.py stub detection so both empty values and verbatim English copies are treated as untranslated stubs, ensuring freshly added keys are picked up for translation.

Enhancements:

  • Add targeted pytest-based unit tests for tools/build utilities, including dictionary parsing, stub detection, and diff-based empty-entry detection.
  • Extend the python-quality workflow to run pytest against tools/build/ as a standalone scripts suite.
  • Document the new i18n guard behavior and rationale in CHANGELOG, ADR 0013, and backlog PRD/tickets, including explicit tracking of existing dictionary debt.

PRD, 3 tickets and ADR for a CI-enforced i18n dictionary guard: block
new MISSING keys unconditionally, block new empty non-EN stubs unless
labeled skip-i18n, and fix translate_i18n.py's stub detection so the
guard is actually satisfiable locally. Grilled with docs 2026-08-10.
Its stub-selection predicate only ever matched a non-EN value
identical to the EN text. generate_i18n_dicts.ps1 -Apply writes a
freshly-added non-EN entry as "" instead, so a new key was never
selected for translation, with or without ANTHROPIC_API_KEY set.

Extract the dict-file parser into a shared tools/build/i18n_dict_utils.py
(also consumed by ticket 03's diff checker), extend the stub predicate
to match "", and add tools/build/'s first test coverage. Wire
`pytest tools/build/` into python-quality.yml so it runs in CI.

Part of FIX-I18N-DICT-GUARD (ADR 0013).
New PR-only CI job i18n-guard, modeled on changelog-guard: covers
tickets 01 and 03 of FIX-I18N-DICT-GUARD.

Unconditional block on a ctld.tr or config-YAML key missing from any
of the four dictionaries - reuses generate_i18n_dicts.ps1's existing
dry-run; STALE stays non-blocking, unchanged.

Diff-scoped block against the PR base on a newly-introduced empty
non-EN entry, via the new tools/build/check_i18n_diff.py. Bypassable
with the skip-i18n label for contributors without local
ANTHROPIC_API_KEY access. Pre-existing debt already on develop, 91 KO
plus 76 ES empty entries, is untouched by this diff-scoped check - a
follow-up lot repays it.

CHANGELOG updated. Closes FIX-I18N-DICT-GUARD, see ADR 0013.
@FullGas1
FullGas1 requested a review from davidp57 as a code owner August 10, 2026 10:04
@sourcery-ai

sourcery-ai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Reviewer's Guide

Fixes i18n stub detection so newly-added empty non-EN entries are recognized, and introduces a CI i18n-guard job plus supporting scripts/tests to block PRs that add missing or newly-empty i18n dictionary entries, with design documented via ADR and backlog tickets.

Sequence diagram for the new i18n-guard CI job

sequenceDiagram
    actor Developer
    participant GitHubCI
    participant generate_i18n_dicts_ps1 as generate_i18n_dicts.ps1
    participant check_i18n_diff_py as check_i18n_diff.py

    Developer->>GitHubCI: Open/ update pull_request
    GitHubCI->>GitHubCI: i18n-guard job starts
    GitHubCI->>generate_i18n_dicts_ps1: run (dry-run)
    generate_i18n_dicts_ps1-->>GitHubCI: output
    GitHubCI->>GitHubCI: search output for MISSING
    alt MISSING found
        GitHubCI-->>Developer: fail PR (missing dict entries)
    else no MISSING
        GitHubCI->>GitHubCI: read skip-i18n label
        alt HAS_SKIP == true
            GitHubCI-->>Developer: pass (empty-stub check bypassed)
        else HAS_SKIP == false
            GitHubCI->>check_i18n_diff_py: python tools/build/check_i18n_diff.py BASE_SHA
            check_i18n_diff_py-->>GitHubCI: exit code
            alt exit code != 0
                GitHubCI-->>Developer: fail PR (newly-empty non-EN entries)
            else exit code == 0
                GitHubCI-->>Developer: pass i18n-guard job
            end
        end
    end
Loading

File-Level Changes

Change Details Files
Refactor i18n dictionary parsing into a shared utility and fix stub detection in translate_i18n.py to treat empty values as untranslated.
  • Extract dictionary entry and __keep_en block parsing into a new i18n_dict_utils module
  • Update translate_i18n.py to use shared parse_dict/parse_keep_en helpers instead of local regex parsing
  • Define a stub as either an empty string or a value equal to the EN text, and centralize stub collection in _collect_stubs
  • Ensure keep_en keys are excluded from stub detection
tools/build/translate_i18n.py
tools/build/i18n_dict_utils.py
Add a CI i18n-guard workflow that enforces presence of i18n keys in all dictionaries and detects newly-empty non-EN entries with an optional skip-i18n bypass.
  • Create i18n-guard job in ci.yml that runs on pull_request with full-history checkout
  • Reuse generate_i18n_dicts.ps1 dry-run output to fail CI when MISSING keys are detected in any dictionary
  • Run check_i18n_diff.py against the PR base SHA to fail CI when new empty non-EN entries are introduced
  • Implement skip-i18n label handling so the empty-entry check can be bypassed while MISSING remains unconditional
.github/workflows/ci.yml
tools/build/check_i18n_diff.py
Introduce unit tests and CI wiring for tools/build scripts to cover parsing and stub/diff detection logic.
  • Add pytest-based tests for dictionary parsing and __keep_en detection
  • Add tests for new-empty-stub diff logic in check_i18n_diff.py
  • Add tests for translate_i18n.py stub classification and collection behavior
  • Extend python-quality workflow to install pytest via pip and run pytest tools/build/
tools/build/test_i18n_dict_utils.py
tools/build/test_check_i18n_diff.py
tools/build/test_translate_i18n.py
.github/workflows/python-quality.yml
Document the i18n-guard design and lot, update backlog and ADR index, and record the change in the changelog.
  • Add ADR 0013 describing the CI-enforced i18n dictionary guard and its trade-offs
  • Add FIX-I18N-DICT-GUARD PRD and tickets describing the lot and its tasks
  • Update backlog README to reference the FIX-I18N-DICT-GUARD lot
  • Add changelog entry summarizing the stub detection fix, new i18n-guard job, and new tooling
dev/adr/0013-ci-i18n-dict-guard.md
dev/adr/README.md
.backlog/FIX-I18N-DICT-GUARD/PRD.md
.backlog/FIX-I18N-DICT-GUARD/tickets/01-ci-missing-key-guard.md
.backlog/FIX-I18N-DICT-GUARD/tickets/02-fix-translate-stub-detection.md
.backlog/FIX-I18N-DICT-GUARD/tickets/03-ci-empty-stub-diff-guard.md
.backlog/README.md
CHANGELOG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Per the repo's default workflow, the README index line is set in the
PR itself rather than as a separate post-merge commit.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In check_i18n_diff.py, _git_show treats any non-zero git show exit status as “file absent”; consider distinguishing between “path does not exist at base” and genuine git errors so you don’t silently ignore repository/CI issues as if they were new files.
  • The new pytest tools/build/ step installs pytest globally with pip alongside the existing Poetry-managed environment; it may be more robust to either reuse Poetry’s venv (e.g. poetry run pytest tools/build/) or constrain the pip install to avoid version drift between the two runners.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `check_i18n_diff.py`, `_git_show` treats any non-zero `git show` exit status as “file absent”; consider distinguishing between “path does not exist at base” and genuine git errors so you don’t silently ignore repository/CI issues as if they were new files.
- The new `pytest tools/build/` step installs `pytest` globally with `pip` alongside the existing Poetry-managed environment; it may be more robust to either reuse Poetry’s venv (e.g. `poetry run pytest tools/build/`) or constrain the pip install to avoid version drift between the two runners.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@FullGas1
FullGas1 merged commit cfb7cd6 into develop Aug 10, 2026
9 checks passed
FullGas1 added a commit that referenced this pull request Aug 10, 2026
…EPAYMENT) (#116)

* fix(i18n): repay pre-existing KO/ES translation debt

Follow-up to FIX-I18N-DICT-GUARD (PR #115), which explicitly deferred
this work. 93 KO / 78 ES empty entries counted at merge; 23 / 8 turned
out to be STALE dead keys in CTLD_i18n_en.lua (no longer referenced by
src/), left untouched. The 70 live entries per language (same key set
for both) are now translated.

No ANTHROPIC_API_KEY was available, so translations were produced and
written directly rather than via translate_i18n.py - the script itself
is untouched, still the documented mechanism for next time.

JTAC and %1 [%2] %3. (no translatable content) added to each
dictionary's __keep_en block rather than left flagged as stubs
forever.

pytest tools/build/ 17/17 green; generate_i18n_dicts.ps1 dry-run
reports 0 MISSING. Lot: .backlog/FIX-I18N-DEBT-REPAYMENT/.

* chore(backlog): mark FIX-I18N-DEBT-REPAYMENT as merged (PR #116)
@davidp57 davidp57 mentioned this pull request Aug 14, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant